Cleaning robot and controlling method thereof

ABSTRACT

A controlling method of a cleaning robot is provided. The controlling method includes obtaining lidar data while a lidar sensor rotates multiple times, calculating a reference value which is the average value of distances from a subject according to angles, based on the obtained lidar data, after the reference value is calculated, comparing the obtained lidar data with the reference value, sensing a motion of an object based on a comparison result, and determining that the object is located in a rotation section where the motion has been detected.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a continuation application, claiming priority under§ 365(c), of an International application No. PCT/KR2021/017200, filedon Nov. 22, 2021, which is based on and claims the benefit of a Koreanpatent application number 10-2021-0003438, filed on Jan. 11, 2021, inthe Korean Intellectual Property Office, and of a Korean patentapplication number 10-2021-0041092, filed on Mar. 30, 2021, in theKorean Intellectual Property Office, the disclosure of each of which isincorporated by reference herein in its entirety.

BACKGROUND 1. Field

The disclosure relates to a cleaning robot for sensing motion of anobject and controlling method thereof.

2. Description of Related Art

A cleaning robot is a device that sucks up debris including dust fromthe floor while autonomously moving in an area to be cleaned withoutuser manipulation.

The cleaning robot is equipped with a lidar sensor to obtain a map ofwhere the cleaning robot is and recognize a location of the cleaningrobot on the map.

Furthermore, the lidar sensor may use cumulative lidar data to detectmotion of an object moving outside the cleaning robot.

In the meantime, the cleaning robot may be connected to a smart phone tocapture the interior of a house through a camera installed therein andprovide the user with a monitoring function.

When there is a companion animal in the house, the user may remotelymove the cleaning robot to monitor a condition of the animal, in whichcase, however, it is difficult to find an initial location of the animaland a series of actions to control movement of the cleaning robot areinvolved with the smart phone, causing inconvenience.

The above information is presented as background information only toassist with an understanding of the disclosure. No determination hasbeen made, and no assertion is made, as to whether any of the abovemight be applicable as prior art with regard to the disclosure.

SUMMARY

Aspects of the disclosure are to address at least the above-mentionedproblems and/or disadvantages and to provide at least the advantagesdescribed below. Accordingly, an aspect of the disclosure is to providea cleaning robot capable of actively monitoring the interior of a houseby using a lidar sensor and controlling method thereof.

Additional aspects will be set forth in part in the description whichfollows and, in part, will be apparent from the description, or may belearned by practice of the presented embodiments.

In accordance with an aspect of the disclosure, a controlling method ofa cleaning robot is provided. The controlling method includes obtaininglidar data with a lidar sensor rotating multiple times, calculating areference value, which is an average of distances to an object forrespective angles, based on the obtained lidar data, comparing lidardata obtained after the reference value is calculated with the referencevalue, and detecting motion of the object based on a result of thecomparing and determining that the object is located in a rotationsection in which the motion is detected.

The controlling method further includes specifying an interested anglearea in a direction toward the object.

The controlling method further includes a camera configured to obtain anexternal image of the cleaning robot, and the specifying of theinterested angle area includes controlling a moving module of thecleaning robot for the camera to capture an area corresponding to theinterested angle area.

The controlling method further includes a sound detector arranged in amain body of the cleaning robot and configured to obtain sound dataabout a sound produced from outside the cleaning robot, and includeidentifying the object by processing the sound data.

The sound detector further includes a plurality of microphonessymmetrically arranged in the main body, and the identifying of theobject includes determining a location of the object based on a medianof signal levels of sound obtained from the plurality of microphones.

The determining of the location of the object includes controlling amoving module of the cleaning robot to capture an area corresponding toa location of the object based on the location of the object determinedbased on sound data obtained through the sound detector.

The determining of the object being located includes specifying aninterested angle area in an angle direction toward the object based ondetection of motion of the object, and in response to a location of theobject determined based on the sound data belonging to the interestedangle area, controlling the moving module to capture an areacorresponding to the interested angle area.

The determining of the object being located includes specifying aninterested angle area in a direction toward the object based ondetection of motion of the object, and in response to a location of theobject determined based on the sound data not belonging to theinterested angle area, controlling the moving module to capture alocation from which sound of the object is detected.

The determining of the object being located includes controlling themoving module to capture a location from which sound of the object isdetected based on the sound data in response to no motion of the objectdetected.

The controlling method further includes a communication moduleconfigured to perform communication with a user equipment, andcontrolling the communication module to provide an event about theobject to the user equipment in response to determination that a soundis produced from the object as a result of processing the sound data.

The controlling method further includes a communication moduleconfigured to perform communication with a user equipment, andcontrolling the camera to capture an area corresponding to theinterested angle area and controlling the communication module toprovide an image of the captured object to the user equipment.

The controlling method further includes a dispenser for storing contentsand discharging the contents, and controlling the dispenser to dischargethe contents at preset intervals in response to detection of motion ofthe object.

The controlling method further includes a laser arranged in the mainbody of the cleaning robot for outputting a laser beam, and controllingthe laser to output the laser beam downward in response to detection ofmotion of the object.

In accordance with another aspect of the disclosure, a cleaning robot isprovided. The cleaning robot includes a main body, a moving moduleconfigured to move the main body, a lidar sensor configured to include alidar light emitter and a lidar light receiver and obtain lidar datawhile rotating multiple times, and at least one processor configured tocalculate a reference value, which is an average of distances to anobject for respective angles, based on the obtained lidar data, comparelidar data obtained after the reference value is calculated with thereference value, and detect motion of the object based on a result ofthe comparing and determine that the object is located in a rotationsection in which the motion is detected.

The cleaning robot further includes a sound detector arranged in themain body of the cleaning robot for obtaining sound data about a soundproduced outside the cleaning robot, and the at least one processoridentifies the object by processing the sound data.

The sound detector includes a plurality of microphones symmetricallyarranged in the main body, and the at least one processor determines alocation of the object based on a median of signal levels of soundobtained from the plurality of microphones.

The at least one processor controls the driver of the cleaning robot tocapture an area corresponding to a location of the object in response todetermination of the location of the object based on the sound dataobtained through the sound detector.

The at least one processor specifies an interested angle area in anangle direction toward the object based on detection of motion of theobject, and in response to the location of the object determined basedon the sound data belonging to the interested angle area, control themoving module to capture an area corresponding to the interested anglearea.

The at least one processor specifies an interested angle area in adirection toward the object based on detection of motion of the object,and in response to the location of the object determined based on thesound data not belonging to the interested angle area, control themoving module to capture a location detected with a sound of the object.

The at least one processor controls the moving module to capture alocation detected with a sound of the object based on the sound data inresponse to no motion of the object detected.

According to the disclosure, various events occurring in the house maybe identified without the user controlling motion of a cleaning robot.Furthermore, according to the disclosure, various conveniences inaddition to a basic function of the cleaning robot may be provided byidentifying a condition of a companion animal in the house in real time.

Other aspects, advantages, and salient features of the disclosure willbecome apparent to those skilled in the art from the following detaileddescription, which, taken in conjunction with the annexed drawings,discloses various embodiments of the disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of certainembodiments of the disclosure will be more apparent from the followingdescription taken in conjunction with the accompanying drawings, inwhich:

FIG. 1 illustrates a cleaning robot, according to an embodiment of thedisclosure;

FIG. 2 is a side view of a cleaning robot, according to an embodiment ofthe disclosure;

FIG. 3 illustrates a lidar sensor of a cleaning robot, according to anembodiment of the disclosure;

FIG. 4 illustrates a light emitter and a light receiver of a cleaningrobot, according to an embodiment of the disclosure;

FIG. 5 is a control block diagram of a cleaning robot, according to anembodiment of the disclosure;

FIG. 6 is a flowchart for describing how a cleaning robot detects motionof an object, according to an embodiment of the disclosure;

FIG. 7 is a diagram for describing a flowchart of FIG. 6 according to anembodiment of the disclosure;

FIG. 8 is a flowchart for describing how a cleaning robot detects motionof an object, according to an embodiment of the disclosure;

FIG. 9 is a flowchart for describing how a cleaning robot detects anobject on a sound basis, according to an embodiment of the disclosure;

FIG. 10 is a diagram for describing a flowchart of FIG. 9 according toan embodiment of the disclosure;

FIG. 11 is a flowchart of a controlling method of a cleaning robot,according to an embodiment of the disclosure;

FIG. 12 illustrates a function provided by a cleaning robot to a userequipment, according to an embodiment of the disclosure;

FIG. 13 illustrates a cleaning robot providing snacks, according to anembodiment of the disclosure;

FIG. 14 illustrates a cleaning robot providing a playtime with laser,according to an embodiment of the disclosure; and

FIG. 15 illustrates a cleaning robot providing a tug play, according toan embodiment of the disclosure.

Throughout the drawings, it should be noted that like reference numbersare used to depict the same or similar elements, features, andstructures.

DETAILED DESCRIPTION

The following description with reference to the accompanying drawings isprovided to assist in a comprehensive understanding of variousembodiments of the disclosure as defined by the claims and theirequivalents. It includes various specific details to assist in thatunderstanding but these are to be regarded as merely exemplary.Accordingly, those of ordinary skill in the art will recognize thatvarious changes and modifications of the various embodiments describedherein can be made without departing from the scope and spirit of thedisclosure. In addition, descriptions of well-known functions andconstructions may be omitted for clarity and conciseness.

The terms and words used in the following description and claims are notlimited to the bibliographical meanings, but, are merely used by theinventor to enable a clear and consistent understanding of thedisclosure. Accordingly, it should be apparent to those skilled in theart that the following description of various embodiments of thedisclosure is provided for illustration purpose only and not for thepurpose of limiting the disclosure as defined by the appended claims andtheir equivalents.

It is to be understood that the singular forms “a,” “an,” and “the”include plural referents unless the context clearly dictates otherwise.Thus, for example, reference to “a component surface” includes referenceto one or more of such surfaces.

The term ‘unit, module, member, or block’ may refer to what isimplemented in software or hardware, and a plurality of units, modules,members, or blocks may be integrated in one component or the unit,module, member, or block may include a plurality of components,depending on the embodiment of the disclosure.

It will be further understood that the term “connect” or its derivativesrefer both to direct and indirect connection, and the indirectconnection includes a connection over a wireless communication network.

The term “include (or including)” or “comprise (or comprising)” isinclusive or open-ended and does not exclude additional, unrecitedelements or method steps, unless otherwise mentioned.

Throughout the specification, when it is said that a member is located“on” another member, it implies not only that the member is locatedadjacent to the other member but also that a third member exists betweenthe two members.

It will be understood that, although the terms first, second, third, orthe like, may be used herein to describe various elements, components,regions, layers and/or sections, these elements, components, regions,layers and/or sections should not be limited by these terms. These termsare only used to distinguish one element, component, region, layer orsection from another region, layer or section.

Reference numerals used for method steps are just used for convenienceof explanation, but not to limit an order of the steps. Thus, unless thecontext clearly dictates otherwise, the written order may be practicedotherwise.

Reference will now be made to embodiments of the disclosure, which areillustrated in the accompanying drawings.

FIG. 1 illustrates a cleaning robot, according to an embodiment of thedisclosure, FIG. 2 is a side view of a cleaning robot, according to anembodiment of the disclosure, FIG. 3 illustrates a lidar sensor of acleaning robot, according to an embodiment of the disclosure, FIG. 4illustrates a light emitter and a light receiver of a cleaning robot,according to an embodiment of the disclosure, and FIG. 5 is a controlblock diagram of a cleaning robot, according to an embodiment of thedisclosure.

Referring to FIGS. 1 and 2 , a cleaning robot 10 in an embodiment mayinclude a main body 11 and a rear cover 12 which form the exterior ofthe cleaning robot 10, a moving module 160 for moving the cleaning robot10, a lidar sensor 110, and a top cover 112 for covering an upper end ofthe lidar sensor 110.

The lidar sensor 110 may be arranged on top of the main body 11 to riseand fall between a first position and a second position at differentheights.

Referring to FIG. 2 , based on the ground on which the cleaning robot 10is located, height H1 of the lidar sensor 110 at the first position ishigher than height H2 of the lidar sensor 110 at the second position.

For example, the lidar sensor 110 may be arranged to be inserted to theinside of the main body 11 or to protrude outward from the main body 11.The lidar sensor 110 may be located in the first position when insertedto the inside of the main body 11, and in the second position whenprotruding outward from the main body 11.

The lidar sensor 110 is inactive while inserted to the inside of themain body 11. However, when the rear cover 12 is implemented in atransparent material, the lidar sensor 110 may be active and is able toperform sensing on an area behind the cleaning robot 10 even in thestate of being inserted to the inside of the main body 11.

The height of the lidar sensor 110 may be controlled based on thecondition of the cleaning robot 1.

In an embodiment of the disclosure, the moving module 160 may includemoving wheels 163 arranged on the left and right sides. For example, themoving wheels 163 may include a left moving wheel arranged on the leftside of the main body 11 and a right moving wheel arranged on the rightside of the main body 11.

In this case, with rotation of the moving wheels 163, the main body 11may move forward, move rearward, or rotate. For example, when both theleft and right moving wheels 163 rotate forward, the main body 11 maymake forward rectilinear motion, and when both the left and right movingwheels rotate rearward, the main body 11 may make rearward rectilinearmotion.

Furthermore, when the left and right moving wheels rotate in the samedirection at different speeds, the main body 11 may make curvilinearmotion to the left or right, and when the left and right moving wheelsrotate in different directions, the main body 11 may rotate to the leftor right at the same place.

Furthermore, the cleaning robot 10 may further include a castor arrangedat the bottom surface of the main body 11. The castor may be installedat the bottom surface of the main body 11 and rotated in the directionof the movement of the main body 11, to assist the main body 11 to movein a stable posture.

Referring to FIGS. 3 and 4 , the cleaning robot 10 may include the lidarsensor 110 for detecting an external object (obstacle).

The cleaning robot 10 may perform cleaning while moving in an area to becleaned, and when an obstacle is detected by the lidar sensor 110, mayavoid or step over the obstacle.

The obstacle detected by the lidar sensor 110 may refer to any objectthat protrudes or is sunken from the floor, wall or ceiling of the areato be cleaned, hindering the moving of the cleaning robot 10. In otherwords, an object located in a moving path of the cleaning robot 10,i.e., in the area to be cleaned, may be an obstacle to the cleaningrobot 10.

For example, not only the furniture, such as a table, a sofa, or thelike, located in the area to be cleaned but also a wall that divides thespace may correspond to the obstacle, and an object, such as a carpet, athreshold or an object, such as a round bar that the cleaning robot 10may climb up and down may also correspond to the obstacle. Furthermore,an object that is not stationarily located in the area to be cleaned,e.g., a glass, a dish, dog droppings, a bag, or the like, may alsocorrespond to the obstacle for the cleaning robot 10 to avoid or stepover.

Whether the detected obstacle is to be avoided or stepped over by thecleaning robot 10 may be determined based on the size of the detectedobstacle, and the size of the obstacle may be determined based on atleast one of height, width and depth of the obstacle.

Furthermore, the cleaning robot 10 may detect a mobile object inaddition to the stationary obstacle with the lidar sensor 110. Theobject is a moving thing in the area to be cleaned, corresponding to aperson or a companion animal.

The cleaning robot 10 may detect motion of the object with the lidarsensor 110 and distinguish the object from the stationary obstacle, andmay find a position of the object with respect to the cleaning robot 10.For example, the cleaning robot 10 may find an azimuth angle of theobject based on the cleaning robot 10 or a cleaning map, and obtain adistance from the cleaning robot 10 and coordinate information of theobject on the cleaning map. A detailed procedure in which the cleaningrobot 10 detects motion of the object and specifies a kind of the objectwill be described later with reference to FIGS. 5 to 10 .

The lidar sensor 110 may include a lidar light emitter 111 for emittinglight, a lidar light receiver 113 arranged to receive light in a presetdirection among reflected light when the light emitted from the lidarlight emitter 111 is reflected from the obstacle, and a circuit board119 to which the lidar light emitter 111 and the lidar light receiver113 are fixed. In this case, the circuit board (or printed circuit board(PCB)) 119 is arranged on a supporting plate 117 that is rotated by arotation driver 115, thereby being able to rotate 360 degrees clockwiseor counterclockwise.

Specifically, the supporting plate 117 may be rotated around an axis Cas a center according to power delivered from the rotation driver 115,and the lidar light emitter 111 and the lidar light receiver 113 arefixed to the circuit board 119 and arranged to be able to rotate 360degrees clockwise or counterclockwise along with the rotation of thecircuit board 119. With this, the lidar sensor 110 may emit and receivelight in 360 degrees to detect an obstacle and an object in alldirections.

The lidar light emitter 111 is a component for emitting light (e.g., aninfrared laser), and may be provided in the singular or plural dependingon the embodiment.

When light emitted from the lidar light emitter 111 is reflected from anobstacle, the lidar light receiver 113 is arranged to receive the lightof a preset direction among the reflected light. An output signalgenerated after the lidar light receiver 113 receives the light may beprovided in an obstacle detection procedure of the controller (notshown).

The lidar light receiver 113 may include a condensing lens forconcentrating the received light and a light sensor for detecting thereceived light. In an embodiment of the disclosure, the lidar lightreceiver 113 may include an amplifier for amplifying the light detectedby the light sensor.

The light sensor may convert the light into an electric signal and sendthe electric signal to the controller. The light sensor may include aphoto diode, a photo diode array, a charge coupled device (CCD) sensor,a complementary metal-oxide semiconductor (CMOS) sensor, a cadmiumsulfide (CDS) sensor, or the like. Available examples of the lightsensor are not, however, limited thereto.

The supporting plate 117 may be arranged to be rotated by the rotationdriver 115. The rotation driver 115 may include an encoder (not shown),which may provide rotation angle information to the controller for anobstacle detection procedure. The rotation angle information of thesupporting plate 117 sent from the encoder may include directioninformation of an obstacle. The controller may detect the obstacle basedon the electric signal output from the lidar light emitter 111 and thelidar light receiver 113 with the rotation angle information of thesupporting plate 117 sent from the encoder of the rotation driver 115.

The circuit board 119 may be arranged on the supporting plate 117 androtated along with the supporting plate 117, and may have the lidarlight emitter 111 and the lidar light receiver 113 arranged on one side.In this case, the lidar light emitter 111 and the lidar light receiver113 may be arranged at different heights in the vertical direction (thez-axis) on the circuit board 119 or at the same height in the verticaldirection (the z-axis) on the circuit board 119.

In another embodiment of the disclosure, the lidar sensor 110 mayinclude the lidar light emitter 111 and the lidar light receiver 113arranged in fixed positions not rotated, and may include a rotatingpolygonal mirror (not shown) or a rotating mirror (not shown) rotated bythe rotation driver 115. In this case, the light emitted from the lidarlight emitter 111 may be radiated in all directions through the rotatingpolygonal mirror or rotating mirror, and the light reflected from anobstacle may be received by the lidar light receiver 113 through therotating polygonal mirror or the rotating mirror.

Structural features of the cleaning robot 10 have thus far beendescribed. Control flows of the cleaning robot 10 detecting motion of anobject and monitoring the object will now be described below.

FIG. 5 is a control block diagram of a cleaning robot, according to anembodiment of the disclosure.

Referring to FIG. 5 , the cleaning robot 10 in the embodiment mayinclude the lidar sensor 110 for detecting external things (obstaclesand objects), a camera 120 for capturing an external viewing area of thecleaning robot 10, a sound detector 130 for detecting a sound producedoutside, a communication module 140 for communicating with a userequipment 20, a controller 150 for controlling the camera 120 and themoving module 160 based on detection of the motion of the object, themoving module 160 for moving the main body 11, a cleaner 170 forperforming cleaning by scattering dust on the floor and sucking up thescattered dust while moving, and a storage 180 for storing various kindsof information required for controlling.

At least one component may be added to or omitted from the cleaningrobot 10 to correspond to the performance of the cleaning robot 10 asshown in FIG. 5 . Furthermore, it will be obvious to those of ordinaryskill in the art that the relative positions of the components may bechanged to correspond to the system performance or structure.

The lidar sensor 110 includes the lidar light emitter 111 for emittinglight to an external object, the lidar light receiver 113 for receivinglight reflected from the external object, and the rotation driver 115for conveying rotation force to the supporting plate 117 on which thelidar light emitter 111 and the lidar light receiver 113 are arranged.

The lidar sensor 110 may control the rotation driver 115 to convey therotation force to the supporting plate 117 under the control of thecontroller 150. In this case, the supporting plate 117 may rotate at aconstant angle speed, and the circuit board 119 fixed on the supportingplate 117 may also rotate at the constant angle speed. With this, thelidar light emitter 111 and the lidar light receiver 113 may identify anexternal object located in the range of 360 degrees by transmitting andreceiving light while rotating at the constant angle speed.

The camera 120 may be arranged in the main body 11 to capture atwo-dimensional (2D) or three-dimensional (3D) image related tosurroundings of the cleaning robot 10. When the controller 150determines a location of the object, the camera 120 may capture an areaincluding the location of the object.

The sound detector 130 is arranged in the main body 11 to detect a soundproduced around the cleaning robot 10. The sound detector 130 mayinclude a plurality of microphones 131 to 134, and the number of themicrophones is not limited to what is shown in FIG. 1 .

The sound detector 130 detects a sound produced around the cleaningrobot 10, and send the sound data to the controller 150 so that thecontroller 150 processes the sound data to identify the object.Specifically, the controller 150 selects a partial section related tothe obtained sound data to extract a feature point in the partialsection. In this case, the controller 150 may determine the featurepoint extracted based on a convolution neural network (CNN) to identifya kind of the object. For example, the controller 150 may process thesound data obtained by the sound detector 130 to determine the objectthat has produced the sound as a companion animal.

Furthermore, the sound detector 130 may be provided as the plurality ofmicrophones 131 to 134 as shown in FIG. 1 , so that the controller 150may determine a location of the object based on the sound data.Specifically, each of the plurality of microphones 131 to 134 may obtainsound data, and the controller 150 may determine a location of theobject by calculating a median of signal levels of the sound.

The communication module 140 may communicate with the user equipment 20.For example, the user equipment 20 is a remote control device to controlmovement of a mobile robot 100 or wirelessly transmit a control commandto perform a task of the mobile robot 100, includes a cellphone or apersonal communications service (PCS) phone, a smart phone, a personaldigital assistant (PDA), a portable multimedia player (PMP), a laptopcomputer, a digital broadcast terminal, a netbook, a tablet, anavigation system, or the like. The communication module 140 may beprovided as a wireless communication module for transmitting orreceiving data with a radio communication protocol already known to thepublic.

The controller 150 may control general operation of the cleaning robot10 and signal flows between the internal components of the cleaningrobot 10, and perform a function to process data. When a command isinput from the user or a preset condition is met, the controller 150 mayrun a program or application stored in a memory (not shown).

The controller 150 may include a processor (not shown), a read onlymemory (ROM) that stores a control program or application forcontrolling the cleaning robot 10, and a random-access memory (RAM) forstoring signals or data input from the outside of the cleaning robot 10or being used as a storage section corresponding to various tasksperformed by the cleaning robot 10.

The controller 150 may determine a location of an object based on lidardata obtained by the lidar sensor 110. Specifically, the controller 150may control the lidar sensor 110 to be driven according to a usersetting or a user input to the user equipment 20 to obtain lidar data,and determine that there is an object in a rotation section having lidardata exceeding a reference value. In this case, when the location of theobject is determined, the controller 150 may control the moving module160 for the camera 120 arranged in the main body 11 to capture an areaincluding the object. Specifically, when a position and direction of thecleaning robot 10 allows the camera 120 to capture the object, thecontroller 150 may capture the object by operating only the camera 120without extra controlling of the moving module 160. When a position anddirection of the cleaning robot 10 is inappropriate for the camera 120to capture the object, the moving module 160 may be controlled to movethe cleaning robot 10 straightly and/or rotationally.

Furthermore, the controller 150 may determine a location and a kind ofthe object based on sound data obtained by the sound detector 130.

Specifically, the controller 150 may process the sound data sent fromthe sound detector 130 to extract a feature point of the sound data,determine the feature point extracted based on the CNN as an inputvalue, and identify the kind of the object based on an output value. Forexample, the controller 150 may process the sound data to figure outthat the object outside the cleaning robot 10 is a companion animal.

As described above, the controller 150 may figure out the location ofthe object by analyzing a plurality of signal levels of the soundobtained from the plurality of microphones 131 to 134 in addition toidentifying the object. The procedure for finding a location of theobject based on the sound data will be described later with reference toFIGS. 9 and 10 .

When detecting motion of an object based on the lidar data, thecontroller 150 may determine a direction in which the object is located,and specify an interested angle area in the direction toward the object.The interested angle area may correspond to an area for monitoring anobject (a companion animal) through the camera 120 equipped in thecleaning robot 10, and an image obtained by capturing at least a portionof the interested angle area may be provided to the user equipment 20.

The controller 150 may determine a location of the object in a differentmethod by processing lidar data obtained by the lidar sensor 110 andsound data obtained by the sound detector 130.

For example, when no motion of the object is detected and only a soundis detected from the object, the controller 150 may determine thelocation of the object based on the sound data.

Furthermore, when both the motion and the sound of the object aredetected, the controller 150 may specify an interested angle area in anangle direction toward the object based on the detection of the motionof the object, and control the camera 120 and/or the moving module 160to capture an area corresponding to the interested angle area when thelocation of the object determined based on the sound data belongs to theinterested angle area specified based on the lidar data.

When the location of the object determined based on the sound data doesnot belong to the interested angle area specified based on the lidardata, the controller 150 may finally determine a location of the objectaccording to an extra priority. This is a case that the object producesa sound as soon as the object is moved. In this case, the controller 150may specify an interested angle area in a direction toward the objectbased on detection of the motion of the object, and in response to thelocation of the object determined based on the sound data not belongingto the interested angle area, control the camera 120 and/or the movingmodule 160 to capture a location detected with the sound of the object.

When a cleaning mode is started, the controller 150 may control themoving module 160 to move and control the cleaner 170 for performingcleaning by scattering dust on the floor and sucking up the scattereddust during the moving.

The cleaner 170 may include a brush module 171 for scattering debrisincluding dust on the floor in an area to be cleaned, and a suctionmodule 173 for sucking up the scattered debris. The brush module 171includes a brush 171 b that rotates to scatter the debris on the floorin the area to be cleaned, and a brush driver 171 a that generates powerprovided to the brush 171 b.

The brush 171 b is arranged at a suction port formed on the bottomsurface of the main body 11, and scatters the debris on the floor in thearea to be cleaned to the inside of the suction port while rotatingaround the rotation axis perpendicular to the forward direction of themain body 11.

The brush driver 171 a may include a brush motor and a driving circuit.A suction module 173 may suck the debris scattered by the brush 171 binto a dust collector, and include a suction fan 173 b that generatessuction force to suck the debris into the dust collector and a suctiondriver 173 a that generates power to rotate the suction fan 173 b. Thesuction driver 173 a may include a suction motor and a driving circuit.

Specifically, when the cleaning mode is started, the controller 150 maytransmit a control signal to the wheel driver 161 for moving andtransmit a control signal to the brush driver 171 a and the suctiondriver 173 a to perform cleaning.

The controller 150 may include at least one memory for storing a programfor carrying out the aforementioned and following operations, and atleast one processor for executing the program. In a case that the memoryand the processor are each provided in the plural, they may beintegrated in a single chip or physically distributed.

The storage 180 may store a control program and control data forcontrolling operation of the cleaning robot 10, and also store acleaning map created based on the data obtained by the lidar sensor 110.For this, the storage 180 may be provided as a known-type of storagemedium.

Components of the cleaning robot 10 and operations of the componentshave thus far been described. Based on the components, a controllingmethod of the cleaning robot 10 will now be described below.

FIG. 6 is a flowchart for describing how a cleaning robot detects motionof an object, according to an embodiment of the disclosure. Theflowchart of FIG. 6 will be described in connection with FIG. 7 .

FIG. 7 is a diagram for describing a flowchart of FIG. 6 according to anembodiment of the disclosure.

Referring to FIGS. 6 and 7 , the controller 150 obtains lidar datathrough the lidar sensor 110, in operation 601. The lidar data is todetect an obstacle and/or an object in all directions by rotating thelidar sensor 110 at a constant speed to emit and receive light in 360degrees. The lidar data includes e.g., information about a location and3D depth of an object based on a time of flight (ToF), which is a timefor light emitted from the lidar light emitter 111 to be received by thelidar light receiver 113 via the object. Accordingly, the controller 150may obtain information about a distance to the object based on aparticular point by rotating the lidar sensor 110 and identify whetherthe object is a stationary object or a moving object.

The controller 150 accumulates a plurality of frames including a portionof the lidar data, in operation 602. Referring to FIG. 7 , thecontrolling method according to an embodiment includes calculating anaverage of the plurality of cumulative frames (C of FIG. 7 ) to set areference value for a stationary obstacle to detect motion of the objectin the rotation section of the lidar sensor 110. In this case, the lidarsensor 110 may obtain lidar data at a resolution of 0.5 degrees, andcreate a plurality of frames having a resolution of 1 degree throughresolution filtering (A->B of FIG. 7 ).

The controller 150 calculates an average of the plurality of cumulativeframes, in operation 603. In this case, the average corresponds to avalue determined due to the stationary obstacle and corresponds to thereference value for detecting motion of the object. The average refersto a distance average between the cleaning robot 10 and the objectobtained by the lidar sensor 110 with the lidar light emitter 111 andthe lidar light receiver 113.

The controller 150 applies secondary filtering to the plurality ofcumulative frames, in operation 604. The applied filtering is windowfiltering for compensating left and right swinging angles from the angleof question, thereby canceling the noise by comparing between an averageof lidar data values in the left and right angles from the angle ofquestion and a lidar data value in the angle of question (D of FIG. 7 ).

In 605, the controller 150 detects motion of the object based on areference value obtained in operations 601 to 604. Specifically, thecontroller 150 detects motion of the object based on a differencebetween the lidar data obtained after accumulating all of the pluralityof frames and the average value, in operation 605. Specifically, thecontroller 150 may calculate the reference value which is an average ofdistances to the object in respective angles based on the lidar dataobtained by rotating the lidar sensor 110, compare lidar data obtainedsubsequently with the reference value, and detect a moving object whenthere is a certain amount of change from the reference value. In anembodiment of the disclosure, the controller 150 may determine thatthere is an object located in a rotation section with lidar dataexceeding the reference value. In this case, the rotation section maycorrespond to the whole or part of section in which the lidar sensor 110is rotated, and corresponds to an angle range of lidar data belonging tothe frame.

When motion of an object is detected in a certain rotation section, thecontroller 150 specifies an interested angle area toward a locationwhere the motion of the object is detected. In the meantime, thecontroller 150 may expand the interested angle area by taking plenty ofmotion of the object into account. Specifically, the controller 150 maydetect motion of an object in a wider rotation section by expanding theangle range of lidar data belonging to the frame. In an embodiment ofthe disclosure, the controller 150 may detect an increase in movingrange of the object based on detection of motion of the object, andexpand the existing interested angle area when the increase in movingrange of the object is detected.

In the disclosure, in addition to the method of FIG. 6 , simultaneouslocalization and mapping (SLAM), which is one of the functions of thelidar sensor 110, may be used to detect motion of an object anddetermine a location of the object.

FIG. 8 is a flowchart for describing how a cleaning robot detects motionof an object, according to an embodiment of the disclosure.

Referring to FIG. 8 , the controller 150 obtains lidar data through thelidar sensor 110, in operation 801. The lidar data is to detect anobstacle and/or an object in all directions by rotating the lidar sensor110 at a constant speed to emit and receive light in 360 degrees.

The controller 150 creates map data based on the lidar data, inoperation 802. Specifically, the controller 150 may create a cleaningmap by accumulating lidar data from all directions in real time.

The controller 150 compares lidar data obtained after creating the mapdata with the map data, in operation 803. In this case, the controller150 may detect motion of the object by detecting a lidar data value inaddition to a fixed value, i.e., the map data, in operation 804.Furthermore, when motion of the object is detected, the controller 150may find a location of the object on the cleaning map and calculate acoordinate value on the cleaning map. In this case, the controller 150may calculate a distance and angle of the object to a current locationof the cleaning robot 10.

When motion of the object is detected, the controller 150 specifies aninterested angle area toward the location where the motion of the objectis detected, in operation 805. In this case, the interested angle areamay be an area having a certain angle range based on an angle toward thelocation of the object.

In the disclosure, the location of the object may be figured out bydetecting a sound of the object in addition to detecting motion of theobject. As described above, the cleaning robot 10 is equipped with thesound detector 130 including the plurality of microphones 131 to 134 todetermine whether the object is a companion animal by identifying thesound produced from the object, and further determine a location of theobject based on signal levels of the sound obtained from the respectivemicrophones 131 to 134.

FIG. 9 is a flowchart for describing how a cleaning robot detects anobject on a sound basis, according to an embodiment of the disclosure.The flowchart of FIG. 9 will be described in connection with FIG. 10 .

FIG. 10 is a diagram for describing a flowchart of FIG. 9 according toan embodiment of the disclosure.

Referring to FIGS. 9 and 10 , the controller 150 obtains sound datathrough the sound detector 130, in operation 901.

The controller 150 identifies the object based on the obtained sounddata, in operation 902. Specifically, the controller 150 selects apartial section related to the obtained sound data to extract a featurepoint in the partial section. In this case, the controller 150 maydetermine the feature point extracted based on the CNN to identify akind of the object. For example, the controller 150 may process thesound data obtained by the sound detector 130 to determine the objectthat has produced the sound as a companion animal.

The controller 150 may calculate a median of the sound signal levels inoperation 903 and determine a location of the object in operation 904.For example, when the sound signal level obtained from the firstmicrophone 131 and the fourth microphone 134 is 10 (a relative value)and the sound signal level obtained from the second microphone 132 andthe third microphone 133 is 8, the median of the signal levels belongsto the above of the 12 o'clock direction. In this case, the location ofthe object belongs to a line extending from a zero point to the median.Furthermore, the controller 150 may calculate an average of signallevels of sound obtained from the plurality of microphones 131 to 134,and figure out a distance between the cleaning robot 10 and the objectbased on the signal level of the sound.

In operation 905, the controller 150 specifies an interested angle areatoward the location of the object, in 805. In this case, the interestedangle area may be an area having a certain angle range based on an angletoward the location of the object.

In the meantime, the cleaning robot 10 according to an embodiment maydetermine a location of the object through sensor fusion between thelidar sensor 110 and the sound detector 130. As described above, theobject may correspond to a moving companion animal, and the companionanimal may make a sound while moving. In this case, the cleaning robot10 may determine a final location of the object by setting prioritiesbetween the lidar sensor 110 and the sound detector 130. This will bedescribed with reference to FIG. 11 .

FIG. 11 is a flowchart of a controlling method of a cleaning robot,according to an embodiment of the disclosure.

Referring to FIG. 11 , the controller 150 determines a first interestedangle area specified based on lidar data in operation 1101, anddetermines a second interested angle area specified based on sound datain operation 1102. The first interested angle area refers to an anglerange to which an object is expected to belong by detecting motion ofthe object through the lidar sensor 110, and the second interested anglearea refers to an angle range to which the object is expected to belongby detecting sound of the object through the sound detector 130. Thefirst interested angle area and the second interested angle area mayeach have a certain angle section, and may be a single angle valueindicating a direction to a particular location.

The controller 150 determines whether the first interested angle areaand the second interested angle area partially correspond to each other,in operation 1103.

When the first interested angle area and the second interested anglearea partially correspond to each other, the controller 150 controls themoving module 160 for the camera 120 to capture an area corresponding tothe first interested angle area, in operation 1104. This is the casethat movement and sound of the companion animal are detected from thesame location, in which case the movement of the companion animal is notbig.

In an embodiment of the disclosure, the controller 150 may specify thefirst interested angle area in an angle direction toward the objectbased on the detection of motion of the object, and in response to thelocation of the object determined based on the sound data belonging tothe first interested angle area, control the moving module 160 tocapture an area corresponding to the first interested angle area. Inthis case, the location determined based on the sound data may bereplaced by the second interested angle area based on the location.

Unlike what is described above, when the first interested angle area andthe second interested angle area do not partially correspond to eachother, the moving module 160 is controlled for the camera 120 to capturean area corresponding to the second interested angle area, in operation1105. This is the case that the companion animal moves relatively a lot,in which case a capturing area of the camera 120 is determined based onthe sound that makes it easy to determine the real-time location of thecompanion animal.

In an embodiment of the disclosure, the controller 150 may specify thefirst interested angle area in a direction toward the object based onthe detection of motion of the object, and in response to the locationof the object determined based on the sound data not belonging to thefirst interested angle area, control the moving module 160 to capture alocation from which the sound of the object is detected. In this case,the location determined based on the sound data may be replaced by thesecond interested angle area based on the location.

Furthermore, in an embodiment of the disclosure, when no motion of theobject is detected, the controller 150 may control the moving module 160to capture a location from which the sound of the object is detectedbased on the sound data. This is the case that the location of thecompanion animal may not be found with the lidar sensor 110 because nomotion of the companion animal is detected, in which case a capturingarea of the camera 120 may be determined only with the sound producedfrom the companion animal.

In an embodiment of the disclosure, the cleaning robot 10 may furtherinclude the communication module 140 for communicating with the userequipment 20. Accordingly, the user may remotely control the cleaningrobot 10 with the user equipment 20 and receive an image of the interiorof the house obtained from the cleaning robot 10. This will be describedwith reference to FIG. 12 .

FIG. 12 illustrates a function provided by a cleaning robot to a userequipment, according to an embodiment of the disclosure.

Referring to FIG. 12 , the user equipment 20 may provide a cleaning mapobtained from the cleaning robot 10 through an application. Furthermore,as described above, the user equipment 20 may found a location of theobject (a companion animal) through the lidar sensor 110 and the sounddetector 130, and control the moving module 160 to move the cleaningrobot 10 to a location suitable for the camera 120 to capture theobject.

When the user activates a monitoring function through an application,the cleaning robot 10 may figure out the location of the object based onthe motion and/or sound of the object so that the camera 120 equipped inthe cleaning robot 10 may capture the object.

In an embodiment of the disclosure, the cleaning robot 10 may furtherinclude the communication module 140 for communicating with the userequipment 20, and the controller 150 may control the communicationmodule 140 to provide an event about the object to the user equipment 20when a sound is determined to be produced from the object as a result ofprocessing the sound data obtained through the sound detector 130. Inthis case, the user may receive the event through the application, andcontrol the cleaning robot 10 for the camera 120 to capture the objectby activating the monitoring function.

Furthermore, in an embodiment of the disclosure, the cleaning robot 10may further include the communication module 140 for communicating withthe user equipment 20, and the controller 150 may control thecommunication module 140 to provide an event about the object to theuser equipment 20 when motion is determined to be detected from theobject as a result of processing the lidar data obtained through thelidar sensor 110. In this case, the controller 150 may control thecamera 120 of the cleaning robot 10 to capture an area corresponding toan interested angle area, and control the communication module 140 toprovide the captured image of the object to the user equipment 20.

In the disclosure, the cleaning robot 10 may interact with the object inaddition to detecting and capturing the object as described above. Forexample, the cleaning robot 10 may detect a touch from a companionanimal, and in response to the touch from the companion animal, output asound that intrigues the companion animal.

FIG. 13 illustrates a cleaning robot providing snacks, according to anembodiment of the disclosure.

Referring to FIG. 13 , the cleaning robot 10 may provide snacks to anobject. For this, the cleaning robot 10 may include a dispenser 13 forstoring and discharging contents. The contents may be snacks for thecompanion animal. The dispenser 13 may be arranged on the side of therear cover 12 of the cleaning robot 10 so that the cleaning robot 10 maythrow the contents while moving forward. In an embodiment of thedisclosure, the cleaning robot 10 may found a location of the object,and control the dispenser 13 to throw the contents while moving in adirection opposite of the object from the perspective of the cleaningrobot 10. In this case, the cleaning robot 10 may throw the contents atpreset time intervals. The user may control the cleaning robot 10 tocontrol a time and interval to throw the contents through a setting forthe user equipment 20.

FIG. 14 illustrates a cleaning robot providing a playtime with laser,according to an embodiment of the disclosure.

Referring to FIG. 14 , the cleaning robot 10 may include a laser 14 toproject a laser beam to reflect off the floor. The laser 14 may bearranged in the main body 11 of the cleaning robot 10. The laser 14 mayoutput a laser beam to reflect off the floor in the shape of a dot, butthe laser beam may have other various shapes according to settings. Forexample, the shape of a butterfly is shown in FIG. 14 , but variousshapes of laser beams may be output based on the kind of the companionanimal. The cleaning robot 10 may induce the object to follow thereflected beam output on the floor by outputting the laser beam whilemoving. In this case, the cleaning robot 10 may figure out through thecamera 120 equipped in the main body 11 how close the object comes tothe reflected beam. When the object comes to the reflected beam at acertain distance or less, the cleaning robot 10 may move to an arbitrarydirection to induce a movement of the object.

FIG. 15 illustrates a cleaning robot providing a tug play, according toan embodiment of the disclosure.

Referring to FIG. 15 , the cleaning robot 10 may include a holder 15 fora tug toy T to be attached to the main body 11. When the companionanimal has the tug toy T in its mouth, the cleaning robot 10 may detectexternal force from the companion animal and control the moving module160 to move the cleaning robot 10 straightly and/or rotationally in adirection opposite of the external force.

Meanwhile, the embodiments of the disclosure may be implemented in theform of a recording medium for storing instructions to be carried out bya computer. The instructions may be stored in the form of program codes,and when executed by a processor, may generate program modules toperform operations in the embodiments of the disclosure. The recordingmedia may correspond to computer-readable recording media.

The computer-readable recording medium includes any type of recordingmedium having data stored thereon that may be thereafter read by acomputer. For example, it may be a read only memory (ROM), a randomaccess memory (RAM), a magnetic tape, a magnetic disk, a flash memory,an optical data storage device, or the like.

The machine-readable storage medium may be provided in the form of anon-transitory recording medium. The term ‘non-transitory recordingmedium’ may mean a tangible device without including a signal, e.g.,electromagnetic waves, and may not distinguish between storing data inthe recording medium semi-permanently and temporarily. For example, thenon-transitory recording medium may include a buffer that temporarilystores data.

In an embodiment of the disclosure, the aforementioned method accordingto the various embodiments of the disclosure may be provided in acomputer program product. The computer program product may be acommercial product that may be traded between a seller and a buyer. Thecomputer program product may be distributed in the form of a storagemedium (e.g., a compact disc read only memory (CD-ROM)), through anapplication store (e.g., Play Store™), directly between two user devices(e.g., smart phones), or online (e.g., downloaded or uploaded). In thecase of online distribution, at least part of the computer programproduct (e.g., a downloadable app) may be at least temporarily stored orarbitrarily created in a storage medium that may be readable to adevice, such as a server of the manufacturer, a server of theapplication store, or a relay server.

While the disclosure has been shown and described with reference tovarious embodiments thereof, it will be understood by those skilled inthe art that various changes in form and details may be made thereinwithout departing from the spirit and scope of the disclosure as definedby the appended claims and their equivalents.

What is claimed is:
 1. A controlling method of a cleaning robot, thecontrolling method comprising: obtaining lidar data with a lidar sensorrotating multiple times; calculating a reference value, which is anaverage of distances to an object for respective angles, based on theobtained lidar data; comparing the lidar data obtained after thereference value is calculated with the reference value; and detecting amotion of the object based on a result of the comparing and determiningthat the object is located in a rotation section in which the motion isdetected.
 2. The controlling method of claim 1, further comprising:specifying an interested angle area in a direction toward the object. 3.The controlling method of claim 2, wherein the cleaning robot includes acamera configured to obtain an external image of the cleaning robot, andwherein the specifying of the interested angle area comprisescontrolling a moving module of the cleaning robot for the camera tocapture an area corresponding to the interested angle area.
 4. Thecontrolling method of claim 1, wherein the cleaning robot includes asound detector arranged in a main body of the cleaning robot andconfigured to obtain a sound data about a sound produced from outsidethe cleaning robot, and wherein the method further comprises:identifying the object by processing the sound data.
 5. The controllingmethod of claim 4, wherein the sound detector comprises a plurality ofmicrophones symmetrically arranged in the main body, and wherein theidentifying of the object comprises determining a location of the objectbased on a median of signal levels of sound obtained from the pluralityof microphones.
 6. The controlling method of claim 5, wherein thedetermining of the location of the object comprises controlling a movingmodule of the cleaning robot to capture an area corresponding to alocation of the object based on the location of the object determinedbased on the sound data obtained through the sound detector.
 7. Thecontrolling method of claim 5, wherein the determining of the objectbeing located comprises: specifying an interested angle area in an angledirection toward the object based on detection of motion of the object;and in response to the location of the object determined based on thesound data belonging to the interested angle area, controlling a movingmodule to capture an area corresponding to the interested angle area. 8.The controlling method of claim 5, wherein the determining of the objectbeing located comprises: specifying an interested angle area in adirection toward the object based on detection of motion of the object;and in response to the location of the object determined based on thesound data not belonging to the interested angle area, controlling amoving module to capture the location from which sound of the object isdetected.
 9. The controlling method of claim 5, wherein the determiningof the object being located comprises controlling a moving module tocapture the location from which sound of the object is detected based onthe sound data in response to no motion of the object detected.
 10. Thecontrolling method of claim 5, wherein the cleaning robot includes acommunication module configured to perform communication with a userequipment, and wherein the method further comprises: controlling thecommunication module to provide an event about the object to the userequipment in response to determining that the sound is produced from theobject as a processing result of the sound data.
 11. The controllingmethod of claim 3, wherein the cleaning robot includes a communicationmodule configured to perform communication with a user equipment, andwherein the method further comprises: controlling the camera to capturethe area corresponding to the interested angle area and controlling thecommunication module to provide an image of a captured object to theuser equipment.
 12. The controlling method of claim 1, wherein thecleaning robot includes a dispenser configured to store contents anddischarge the contents, and wherein the method further comprises:controlling the dispenser to discharge the contents at preset intervalsin response to detection of motion of the object.
 13. The controllingmethod of claim 1, wherein the cleaning robot includes a laser arrangedin a main body of the cleaning robot and configured to output a laserbeam, and wherein the method further comprises: controlling the laser tooutput the laser beam downward in response to detection of motion of theobject.
 14. A cleaning robot comprising: a main body; a moving moduleconfigured to move the main body; a lidar sensor configured to include alidar light emitter and a lidar light receiver and obtain lidar data asit rotates multiple times; and at least one processor configured to:calculate a reference value, which is an average of distances to anobject for respective angles, based on the obtained lidar data, comparelidar data obtained after the reference value is calculated with thereference value, detect motion of the object based on a result of thecomparing, and determine that the object is located in a rotationsection in which the motion is detected.
 15. The cleaning robot of claim14, further comprising: a sound detector arranged in the main body ofthe cleaning robot and configured to obtain a sound data about a soundproduced from outside the cleaning robot, wherein the at least oneprocessor is further configured to identify the object by processing thesound data.